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MOVIE ENHANCEMENT 


BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to multimedia. More specifically, the present 
invention discloses a method of enhancing movies, video, or other multimedia in 
order to provide users with a more rewarding viewing experience. 
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Description of the Prior Art 

Film is a popular fonn of art or entertainment. The affect on movie-goers is 
dramadc when movies are viewed in a cinema or movie flieater. When movies an= 
made or reproduced, care is tal.e„ to ensure that the lighting, brighmess. and contrast 
15 of the film is suitable for the dim or dark theater settmg. However, problems result 
when these movies are transferred from film to digital formats such as video compact 
disc (VCD) or digital video disc (DVD). 

It is a common problem that movies look too dark when watched on 
televisions (TV), cathode ray tube (CRT) monitors, or liquid crystal display (LCD) 
20 panels. Because many movies are intended for playback in a relatively dark 
enviromnent, such as a movie theater, video usually appears to be lacking in 
brightness and contrast when played back in a relatively well-lit enviromnent, such as 
regular room lighting. A conventional way to compensate for this problem is to 
change the display device's brightness level and/or gamma correction strength. 
25 However, an increase in brightness will typically result in a lack of blackness. 
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Additionally, an increase in gannna will typically result in .00 li«le conttas, in brigh, 

regions and too much contrast in dark regions of the movie. 

Refer .0 Figure 1. which is a graph illustrating the effect of increasing 
brighmess of a video signal. As shown in Figure ,. a« origimU brightness level 
represented as a straight line from 0 to 255. In contrast as shown by the adjusted 
output line, increasing the brightness results in a more dramatic increase in brighmess 
of darker regions than in areas which are already relatively bright. As a resul, the 
darkest output in no longer black enough and the overall video output appear, washed 
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out. 
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Refer to Figure 2, which is a graph illustrating the effect of increasing gamma 
of a video signal. In Figure 2. the original gamma is represented as a straight line 
from 0 to 255. By increasing the gamma, the output which is brighter ends up having 
too little contrast and the output which is darker ends up having too much contrast. 
Therefore, the picture quality is not optimal. 

Therefore, there is need for an improved method of enhancing movies or video 
which results in a high quality video output and provides viewers with a rewarding 
viewing experience. 
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SUMMARY OF THE INVENTION 
To achieve these and other advantages and in order to overcome the 
disadvantages of the conventional method in accordance with the purpose of the 
invention as embodied and broadly described herein, the present invention provides a 
movie enhancement method which corrects the video image so that the brightness and 
contrast levels are proper when the video is viewed in a well-lit environment 

In order to achieve a closer-to-movie theater viewing experience, the present 


2 


.nvention p^vides a „e,hod of enhancing video or movie. The enhaneemen. meftod 
can be applied ,o YUV or o*er video fonnats which a. used for TV or digiia, n,edia 
Additionally, fte me*od can be apphed ,o any video forma., snch as HSV. RGB, etc. 
that can be converted or transformed into YUV. 
5 In embodiments of the present invention, the method provides a curve with a, 

least one inflection point such that at least one region has a concave upwanl a., and 
another region has a concave downward arc. 

In an embodiment, the improved curve provides relatively less contrast in 
relatively dark regions and relatively more contrast in relatively bright regions. By 
1 0 taking into account the visual sensitivity to various luminance levels, a neutral point is 
selected to be located at a relatively dark point. To the darker side of the neutral point, 
luminance is suppressed. To the brighter side of the neutral point, luminance is 
enhanced. This improved curve is termed a "luminance-mapping" curve and is 
applied to the luminance (Y) signal so as to enhance both brightness and contrast. 
15 In anther embodiment, the luminance-mapping curve is adaptively adjusted 

over time according to the average luminance level of the input signal. By taking into 
account the effect of the luminance mapping over the avemge luminance level and the 
fact that the luminance level varies over time, the luminance-mapping curve can be 
shifted in order to keep the video with good contrast and preserve the average 
20 luminance level after luminance mapping through time. This is achieved by 
measuring the mean and variance of the luminance signal and generating an 
adjustment to the luminance-mapping curve according to the measurements. 

In order to preserve the color saturation level, the chrominance signals (UA^) 
can also be adjusted according to the change in Y. 
!5 To improve the color saturation level, a chrominance-mapping curve is 
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designed to in^prove the contrast of color U/V components. In an embodiment, this 
curve is configured to have a neutral point at the mid-point. 

In other embodiments, movie enhancement can be applied to regions. Regions 
can be overlapped and each can have its own variation of enhancement. TTierefore, 
5 movie enhancement can be used with object detection and image segmentation 
algorithms. It can also be used with MPEG4. In this case, movie enhancement can be 
used with background or front-objects. In other cases, it can be used with the original 
movie content and subtitles can be blended later. 

In other embodiments, movie enhancement can also be a temporal filter. For 
10 interlaced content, movie enhancement can be applied to each field. 

Some embodiments may introduce coarse edges due to high contrast in certain 
luminance levels. In these cases, dithering can be used to reduce the efiect. 

Other areas which can utilize the movie enhancement method are flickering 
compensation for florescent light, scene change detection, fade detection, used with 
15 other special effects in which subjects change with time, and used in another color 
space, like HSV, to change tone of color. 

These and other objectives of the present invention will become obvious to 
those of ordinary skill in the art after reading the following detailed description of 
preferred embodiments. 

20 It is to be understood that both the foregoing general description and the 

following detailed description are exemplary, and are intended to provide fiuther 
explanation of the invention as claimed. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings are included to provide a further understanding 
of the invention, and are incorporated in and constitute a part of this specification. 
The drawings illustrate embodiments of the invention and, together with the 
5 description, serve to explain the principles of the invention. In the drawings. 

Figure 1 is a graph illustrating the effect of increasing brightness of a video 

signal; 

Figure 2 is a graph illustrating the effect of increasing gamma of a video signal; 
Figure 3 is a graph illustrating a luminance-mapping curve according to an 
1 0 embodiment of the present invention; 

Figure 4 is a graph illustrating a contrast-luminance adjustment curve 
according to an embodiment of the present invention; 

Figure 5 is a graph illustrating a final enhancement mapping curve according 
to an embodiment of the present invention; 
15 Figure 6 is a graph illustrating a U curve according to an embodiment of the 

present invention; and 

Figure 7 is a graph illustrating a V curve according to an embodiment of the 
present invention. 


20 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Reference will now be made in detail to the preferred embodiments of the 
present invention, examples of which are illustrated in the accompanying drawings. 
Wherever possible, the same reference numbers are used in the drawings and the 
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description to refer to the same or like parts. 

In an embodiment of the present invention, the movie enhancement method is 
adjustable through two parameters: intensity and variation. These parameters can be 
set by the user according to the user's personal preference. Y, U, V are scaled to have 
5 a normal range of 0-255 with a center at 128. It will be understood by one of 
ordinary skill in the art that other ranges can also be applied in similar formats. This 
embodiment has been implemented with InterVideo Inc.'s WinDVD which is the 
most popular PC-based software DVD player in the world. WinDVD has also 
designed a dedicated UI to control these parameters. 
10 Refer to Figure 3, which is a graph illustrating a luminance-mapping curve 

according to an embodiment of the present invention. 

Intensity is used to control the amount of adjustment applied to Y. In an 
embodiment, the change is one-directional, a value of 0 corresponds to no effect, and 
a value of 100 corresponds to maximum effect. It will be understood by one of 
1 5 ordinary skill in the art that other scales and ranges can also be used. 

The luminance-mapping transfer function is F() , which is a 1-to-l mapping of 
the Y signal. One embodiment of the luminance-mapping curve is configured to 
increase luminance on the bright side of a neutral point, and to decrease luminance on 
the dark side of the neutral point. Advantageously, the neutral point can be located at 
20 a relatively dark point of the luminance range to address the issue of visual sensitivity 
to various luminance levels: 

r= F(Y) 

hitensity is i, which is one of the control elements and has a value between 0 
25 and 100. The Iviminance mapping based on intensity is adjusted: 
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F(Y, i) = Y + - r) * /• / 1 00 , where 
F(Y,\00) = F{Y), F(Y,0) = Y 

Refer to Figure 4, which is a graph illustrating a contrast-luminance 
adjxistment curve according to an embodiment of the present invention. 

GQ is a "luminance-contrast adjustment" transfer function, which is closely 
related to FQ and, in one embodiment, generates an adjustment to the luminance level 
based on the mean luminance signal of the video. GQ moves the input signal toward 
the center of FQ which has relatively high contrast. Also, GQ helps keep the average 
luminance at a constant level at the output. In one embodiment, GQmd FQhave the 
same neutral point location, where the luminance level is not changed. 

The luminance-contrast adjustment can be further weighted by the variance of 
the luminance signal. Signals with relatively large variance typically use less 
luminance-contrast adjustment, and signals with relatively small variance typically 
use more luminance-contrast adjustment. Let sqrtQ be the square-root operator, 
be the variance of Y signal, and Y^^ be the maximum range of Y. In one embodiment, 
the following formula is used to provide a weighting in the range of about 0.5-2.0. 

(2*Y^ -sqrt{Y^))KsqrtiY^) + Y^) 

To provide consistent contrast and average luminance over time, one 
embodiment measures the average Y^ and variance Y^ of the luminance signal for 
each video frame. It is understood that in other embodunents, the average and 
variance Y^ can be computed from selected frames, such as, for example, every other 
frame or every second. 


g = G(Y^y (2 * y_ - sqrt(Y^))lisqrt(X^) + ) 

Refer to Figvire 5, which is a graph illustrating a final enhancement mapping 
curve according to an embodiment of the present invention. 

The luminance-contrast adjustment based on control factor intensity is g(S), 

Thus for each video frame, based on input luminance and control intensity, the 
final enhancement mapping transfer fimction FG() for the luminance signal (Y), is 
obtained. 

FG{Y,i) = FiY + g(i),i) 
There can be some hard-clippings at both ends of the FGQ curve after the 
luminance-contrast adjustment. These ends of the FGQ curve can be re-interpolated 

to provide a soft-clipping effect. 

To preserve the color saturation level, the U/Y signal can be adjusted as 

follows: 

C/^=(f/-128)*n„,/y + 128 
F^,=(K-128)*y„„,/r + 128 

Additionally, the actual calculation can be simplified to avoid division in 
implementation through fixed-point computation. 

Variation is used to control the color saturation level of the video signal. The 
following process enhances the contrast of the chrominance signal (UIV) with the 
neutral point located at the mid-point, which indicates "no color" in the YUV format. 
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In other embodiments that intend to offset color, the neutral point for the 
chrominance-mapping transfer function can be offset from the mid-point. 

Refer to Figure 6, which is a graph illustrating a U curve according to an 
embodiment of the present invention and to Figure 7, which is a graph illustrating a V 
5 cvirve according to an embodiment of the present invention. 

FUQ and FVQ are the chrominance-mapping transfer functions, which 
enhance the contrast of chrominance signals. 

UOorh Ulorh VOorh and Vlori are pre-defined constants that can be tailored to 
different viewing conditions, such as indoor and/or outdoor. In one embodiment, the 
10 mapping curve is weighted by the variance of the chrominance signals U^^ and V^^ . 
Signals with relatively large variance typically use less chrominance adjustment, and 
vice versa. U^^ and are the maximum range of U and V: 

UO^ =C/0,„ X2^U^ -sqrt{U^))l{sqrt{U^) + U^^) 
15 C/1, =m,„. %2^U^ -sqrt(JJ^))l{sqrt{U^^)^U^^) 

FO, = F0,„. *(2 * V^^ - sqrt{V^^ )) l{sqrt{V^^ ) + V^^^ ) 
= Fl,„. %2 * K,3x - sqrt{V^))lisqrt(y,^)^ V^^ ) 

Let V be the variation ranging from 0 to 100, UO, Ul, VO, VI change linearly 

20 with v: 

m = m, *v/ioo 

Fl = Fl,*v/100 
(70 = (/0, *v/100 
25 FO = KO, *v/100 

Variation in color saturation can be achieved by changing the trajectory of C/ft 
UL VO, and VI. 

Following is a description of details regarding implementation of the movie 
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enhancement method of the present invention. 

Let Y, U, and V be the input signal. One embodiment perforais the following 
operations for every frame. 

1 . Obtain user-input intensity / and variation v. 

2. Calculate average and variances 

3. Calculate luminance-contrast adjustment gii) from i, , and . 

4. Construct FG(i) curve from FQ curve, /, and g(i) . 

5. Construct FUQ and FVQ curve from v, and . 

In other embodiments, operations 1-5 described above can be performed on a 
less frequent basis than every frame to reduce the computation complexity. For 
example, Y^ , , can be calculated when Y^^ has a significant change. 
Operations 3 and 4 can be performed when 7^g has a significant change and upon a 
change in user-input intensity /. Operation 5 can be performed when and 
have significant changes and upon a change in user-input variation v. 

To reduce the computation complexity, all cvirves F() .GO, FG(,/) , FUQ, and 
FVQ can be realized as a fixed-point lookup table, and the mapping calculations can 
be realized by table watching. In other embodiments, a 3"^ order or higher polynomial 
is used to emulate those curves. 

In another embodiment of the present invention the following operations are 

performed for every pixel: 

6. Get the output luminance signal Youi = FG(Y,i) . 

7. Get the output chrominance signal U'=FU(U) and V'=FU(U). 
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8. Restore color saturation from Y-adjustment: let dY = Y^, - Y , in one 
embodiment, Y^„,/Y is simplified as (128 + ^/7)7128, thus 

^om =(C/*-128)*(128 + f/r)/128 + 128 
Ku, =(K'-128)*(128 + i/y)/128 + 128 

It will be apparent to those skilled in the art that various modifications and 
variations can be made to the present invention without departing from the scope or 
spirit of the invention. In view of the foregoing, it is intended that the present 
invention cover modifications and variations of this invention provided they fall 
within the scope of the invention and its equivalent. 
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